{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "7d5b8b60-caec-4d8b-9abc-fef538bfebc2",
   "metadata": {},
   "source": [
    "# 深度学习网络实验——以VGG和ResNet为例"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4034790a-7ec3-40bc-9a51-34138e1cf8f1",
   "metadata": {},
   "source": [
    "# 一、 VGG网络介绍"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d71e3006-89c6-4713-86d4-e31a47c41206",
   "metadata": {},
   "source": [
    "<font size=4> VGG网络是经典的卷积神经网络之一，主要的思想是用连续的多个$3\\times3$的卷积核代替单层的$5\\times5$、$7\\times7$卷积核，从而达到减少网络参数、加深网络深度的目的。根据不同的用法，VGG网络有着不同的网络结构，如下图所示："
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3cfb03e7-81d9-4d9f-9f81-8447783b73f1",
   "metadata": {},
   "source": [
    "![网络结构参数](pic/1.png)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "93dcc00d-c685-4001-b6fc-d0dff44806d7",
   "metadata": {},
   "source": [
    "<font size=4> 图片中，不同的列表示不同规模的VGG网络各层结构。以第D列为例，这一列的参数表示了VGG16网络的参数，包括通道数从64到512的13个卷积层、3个全连接层和穿插其中的最大池化层等。具有可训练参数的网络层是卷积层和全连接层，一共有16个，所以叫做VGG16网络。在搭建网络中，还会在卷积层和全连接层后添加归一化和激活函数，这是神经网络的常规操作，因此图片中并不展示出来。VGG16的网络结构可以想象成这个样子："
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d159ea56-3366-43c8-a802-ae9b2e41d58c",
   "metadata": {},
   "source": [
    "![网络结构](pic/3.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ce92dda3-f5e7-41de-8cef-6ac7ce204432",
   "metadata": {},
   "source": [
    "# 二、ResNet网络介绍"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dfe59e15-dd5a-474f-844d-7fcb08518b75",
   "metadata": {},
   "source": [
    "<font size=4>ResNet网络也是经典的卷积网络之一。与VGG中笔直的数据流不同，在ResNet中，独有着“残差连接”这个概念。在某些层中，网络会将未经运算的数据直接加到经过运算的结果上，这部分数据就叫做“残差”。因此，ResNet与其他卷积神经网络相比，更注重了线性变换与非线性变换的平衡，在各种任务中取得了很好的效果，是现在最常用的神经网络之一。常见的ResNet网络有着不同的规格，如ResNet18、ResNet34、ResNet50等。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6b71a230-4db5-4e7d-9aab-770835b5db46",
   "metadata": {},
   "source": [
    "![网络结构参数](pic/2.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c1c8c9da-7576-4c5c-aa20-b7df387597ff",
   "metadata": {},
   "source": [
    "# LSTM介绍\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cc7f422e-8498-4aed-8046-971742f46ae9",
   "metadata": {},
   "source": [
    "![网络结构参数](pic/7.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "91918edd-52f5-40f4-9dfc-7bb995154d5d",
   "metadata": {},
   "source": [
    "<font size=4>LSTM（Long Short-Term Memory，长短时记忆网络）是一种特殊的循环神经网络（RNN），用于处理并记住长期的序列信息。与传统 RNN 不同，LSTM 在设计上可以更有效地捕捉长距离的依赖关系，防止梯度消失或爆炸的问题。\n",
    "\n",
    "LSTM 结构\n",
    "LSTM 由以下几部分组成：\n",
    "\n",
    "输入门（Input Gate）： 控制新输入的信息进入当前状态的多少。\\\n",
    "遗忘门（Forget Gate）： 控制当前状态的哪些部分需要被保留或遗忘。\\\n",
    "输出门（Output Gate）： 控制当前状态中的哪些部分将作为当前时间步的输出。\\\n",
    "细胞状态（Cell State）： 长期记忆信息的传递，负责保留序列信息。\\\n",
    "LSTM 的工作过程\\\n",
    "遗忘门：\\\n",
    "接受前一个时间步的隐藏状态和当前时间步的输入。\\\n",
    "通过激活函数 Sigmoid，输出值范围在 [0, 1] 之间，用来控制当前细胞状态的遗忘程度。\\\n",
    "输入门：\\\n",
    "接收输入和前一个隐藏状态。\\\n",
    "通过 Sigmoid 产生控制信号，同时产生新候选记忆信息。\\\n",
    "细胞状态更新：\\\n",
    "根据遗忘门和输入门的输出，更新细胞状态。\\\n",
    "输出门：\\\n",
    "控制当前细胞状态中要输出的部分，作为当前隐藏状态的输出。\\\n",
    "LSTM 的优点\n",
    "记忆能力： 可以长期记忆序列信息。\n",
    "抗梯度消失能力： 在长时间序列的情况下能够保持较好的学习效果。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b147f601-6286-4462-b157-fcac486d1a65",
   "metadata": {},
   "source": [
    "# 三、神经网络的搭建"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "96c3dc23-f675-477a-92cf-fe3e69faa6c8",
   "metadata": {},
   "source": [
    "<font size=4>在MindSpore框架中，一个神经网络的搭建一般遵循这样的模板："
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6f9bf05e-10ef-4c8c-b86c-e7ed51cf904c",
   "metadata": {},
   "source": [
    "import mindspore.nn as nn\n",
    "\n",
    "\n",
    "class Net(nn.Cell):\n",
    "    def __init__(self, Args):  # 必要\n",
    "        super(Net, self).__init__()\n",
    "        \"\"\" 在这里定义所需要的网络层\n",
    "        ...\n",
    "        ..\n",
    "        .\n",
    "        \"\"\"\n",
    "\n",
    "    def block(self, Args):  # 不必要\n",
    "        \"\"\" 若网络结构具有重复性，可以在这种函数中将重复的几个层打包成一个块，在上面的函数里调用，避免繁杂的编写\n",
    "        ...\n",
    "        ..\n",
    "        .\n",
    "        \"\"\"\n",
    "\n",
    "    def construct(self, x):  # 必要\n",
    "        \"\"\" 在这里使用定义好的网络层构建网络\n",
    "        ...\n",
    "        ..\n",
    "        .\n",
    "        \"\"\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "35b0423f-4295-4e78-91da-7fd5b43d71e2",
   "metadata": {},
   "source": [
    "<font size=4>本例程通过搭建简化的VGG网络：VGG7来具体描述这个模板（参照VGG7.py）。在VGG中，每个卷积层后面都会跟随归一化和激活函数，因此，如果把这三个部分写成一个网络层，就可以让代码更加精简清爽，方便复杂网络的搭建。如VGG7.py中的代码所示："
   ]
  },
  {
   "cell_type": "markdown",
   "id": "346603e6-fa38-46fd-ad7d-8a3e6ac8825f",
   "metadata": {},
   "source": [
    "def conv_block(self, in_channel, out_channel):\n",
    "    seq = nn.SequentialCell(\n",
    "        [\n",
    "            nn.Conv2d(in_channel, out_channel, 3, padding=1, pad_mode='pad'),\n",
    "            nn.BatchNorm2d(out_channel),\n",
    "            nn.ReLU()\n",
    "        ])\n",
    "    return seq"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bac5be38-7b9d-452c-b9f6-359c62ffee72",
   "metadata": {},
   "source": [
    "<font size=4>nn.SequentialCell() 函数可以提供一个封装的功能，将参数列表里的多个网络层按顺序封装成一个层。调用这个函数，就会按顺序产生Conv2d、BatchNorm2d、ReLu三个网络结构，好比将三块小积木首尾拼接成一个稍大的积木一样。这样，在__init__() 函数中，就可以调用这个函数搭建网络。类似的，这样构建的网络层也可以再次成为nn.SequentialCell() 的参数，参与构建更大的网络层，好比用稍大的积木拼接出更大的积木一样。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "48b86a4c-73a9-41ef-9097-fab408494146",
   "metadata": {},
   "source": [
    "<font size=4>这种写在class里的网络块结构也经常被拆出来，作为另一个class来定义。注意观察上面class的定义，只要在class定义的第一行括号里，继承上nn.Cell属性，就可以被视为一种网络结构，既可以被训练，也可以被调用来组成更大的网络。在本例程的ResNet12.py文件中，就是将组成大网络ResNet12的基本单元ResBlock作为一个小的网络，单独用一个class定义的。ResNet12由四个残差块（就是基本单元）组成，每个残差块的结构都一样，由3层卷积层组成。根据相关文献，每个残差块的结构如下图所示："
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f624bbf6-827a-4c7b-8ab8-b9b58c39f1dd",
   "metadata": {},
   "source": [
    "![网络结构](pic/4.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e2b5f3bb-930c-4d64-990d-b4976cdb918c",
   "metadata": {},
   "source": [
    "因此，在Resnet12.py文件中，残差块的结构定义如下："
   ]
  },
  {
   "cell_type": "markdown",
   "id": "417dccf8-4e0a-450b-a598-821be5a9f912",
   "metadata": {},
   "source": [
    "class ResBlock(nn.Cell):\n",
    "    def __init__(self, in_channel, out_channel):\n",
    "        super(ResBlock, self).__init__()\n",
    "        self.seq = nn.SequentialCell(\n",
    "            [\n",
    "                nn.Conv2d(in_channel, out_channel, 3, stride=1, padding=1, pad_mode='pad'),\n",
    "                nn.BatchNorm2d(out_channel),\n",
    "                nn.ReLU(),\n",
    "                nn.Conv2d(out_channel, out_channel, 3, stride=1, padding=1, pad_mode='pad'),\n",
    "                nn.BatchNorm2d(out_channel),\n",
    "                nn.ReLU(),\n",
    "                nn.Conv2d(out_channel, out_channel, 3, stride=1, padding=1, pad_mode='pad'),\n",
    "                nn.BatchNorm2d(out_channel),\n",
    "            ])\n",
    "        self.shortup = nn.SequentialCell(\n",
    "            [\n",
    "                nn.Conv2d(in_channel, out_channel, 1, stride=1),\n",
    "                nn.BatchNorm2d(out_channel),\n",
    "            ])\n",
    "        self.relu = nn.ReLU()\n",
    "        self.max_pool = nn.MaxPool2d(kernel_size=2, stride=2)\n",
    "\n",
    "    def construct(self, x):\n",
    "        y = self.seq(x) + self.shortup(x)\n",
    "        y = self.relu(y)\n",
    "        return self.max_pool(y)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "210e9b7a-c00e-4190-8330-6e0a72e14772",
   "metadata": {},
   "source": [
    "<font size=4>construct() 函数定义网络运行时数据是如何流过网络的。这个函数的输入一般是数据，比如图片数据张量和时序数据张量等。在较为简单的网络中，数据一般按顺序经过不同的网络层，最终得到网络的结果。如VGG7.py中的代码所示："
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d1eb28fb-1d4f-40b1-b9df-639689e1e1e7",
   "metadata": {},
   "source": [
    "def construct(self, x):\n",
    "    # 使用定义好的运算构建前向网络\n",
    "    x = self.conv1(x)\n",
    "    x = self.max_pool2d(x)\n",
    "\n",
    "    x = self.conv2(x)\n",
    "    x = self.max_pool2d(x)\n",
    "\n",
    "    x = self.conv3(x)\n",
    "    x = self.max_pool2d(x)\n",
    "\n",
    "    x = self.conv4(x)\n",
    "    x = self.max_pool2d(x)\n",
    "\n",
    "    x = self.conv5(x)\n",
    "    x = self.max_pool2d(x)\n",
    "\n",
    "    x = self.flatten(x)\n",
    "    x = self.fc9(x)\n",
    "    x = self.relu(x)\n",
    "    x = self.fc10(x)\n",
    "    x = self.relu(x)\n",
    "    x = self.fc11(x)\n",
    "    return x"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fe494287-2f2d-4d9f-98c3-1be9c2c410f8",
   "metadata": {},
   "source": [
    "<font size=4>输入数据x依次经过定义的conv1、maxpool2d、conv2直到fc11等网络层后，得到网络的输出结果。若网络执行分类任务，那么将这些结果拿去经过softmax等处理后，就可以得到分类标签。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "69bddbd7-38ad-4f8d-b5b4-3a87ba481167",
   "metadata": {},
   "source": [
    "<font size=4>将神经网络视作一个函数，那么我们输入图片、时序等数据，就可以得到分类预测标签、未来预测趋势等结果。这是网络的前向过程。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7ee71b2a-6519-45fd-845f-40595799936c",
   "metadata": {},
   "source": [
    "<font size=4>在训练阶段，根据预测结果和真实结果的差距，可以计算出网络训练的损失。通过梯度反向传播技术，可以根据损失修改网络的参数，从而使网络的结果更接近我们想要的结果。这是网络的训练过程。我们可以设置训练的参数，例如训练轮数等，来让网络训练到让人满意为止。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3e1a0763-8cac-4717-aa54-3455b1380ec6",
   "metadata": {},
   "source": [
    "# 四、神经网络的训练"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a3f4aa26-bc5b-4926-bdb8-1adfd2c2baab",
   "metadata": {},
   "source": [
    "<font size=4> 神经网络的训练，必然包括网络定义、数据集定义、训练脚本的编写这些部分。关于数据集的定义，这里不再赘述。关于训练脚本的编写，对于不同的深度学习框架，有着不同的编写结构；即便同一个框架，也有着不同的写法。对于MindSpore框架，可以调用其内部定义的高级API来实现网络的训练。具体的写法可以参考main.ipynb里的train()函数。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d35e41be-0b8f-4d58-957f-f43a853bab19",
   "metadata": {},
   "source": [
    "<font size=4> 在main.ipynb中，已经写好了数据集部分和训练的脚本。train() 函数具有两个参数，第一个参数控制网络采用哪种结构，第二个参数控制训练的轮数。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "be256eef-6990-435f-b72b-40b81bfc235f",
   "metadata": {},
   "source": [
    "train(\"vgg7\", 1)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "accfc78e-e923-4b9f-a499-54230a65e3b9",
   "metadata": {},
   "source": [
    "<font size=4> 这行代码就表示，将网络定义为vgg7网络，并训练1轮。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8a91a3fb-5322-497e-b044-e8f67f39d9e9",
   "metadata": {},
   "source": [
    "# 五、实验内容"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "30a7d121-93f0-49b6-a946-d694e3df37eb",
   "metadata": {},
   "source": [
    "## 任务一 对照第一章中的VGG11参数，补全残缺的VGG11.py文件"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ee3044a3-7374-43fa-84b6-950551fdb13e",
   "metadata": {},
   "source": [
    "<font size=4> 本例程已经包含VGG7和ResNet12两个神经网络的样例。请注意观察本例程第一章中VGG网络结构参数的图片，VGG7就是将VGG11中第3、4、5个卷积阶段中重复的层去掉，并去掉一个全连接层实现的。一共去掉了4个层，因此从VGG11变为了VGG7。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "033e389a-14f1-47a8-9f89-f879a5fd51df",
   "metadata": {},
   "source": [
    "<font size=4> 可以参考和复制VGG7.py文件的代码，对缺少的层进行补全。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b1364898-4f85-46e9-9fcc-dd9fbc6ea74c",
   "metadata": {},
   "source": [
    "## 任务二 对照下面的网络结构，补全残缺的Resnet18.py文件"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "801815bf-c78f-4856-9496-592d92e5b7a6",
   "metadata": {},
   "source": [
    "<font size=4> Resnet18中有两种不同的残差块，它们的结构如下所示："
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2bec6f71-9af2-4528-a996-fb9208fcf3c1",
   "metadata": {},
   "source": [
    "![网络结构](pic/5.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6f57a115-b452-4b64-9410-91c4a874f3b0",
   "metadata": {},
   "source": [
    "<font size=4> 在Resnet18.py文件中，左边的残差块被ResBlock实现，右边的残差块被ResBlockDS实现。请参考Resnet12中对残差块的实现过程和代码，对以上两个残差块的代码进行补全。下图是resnet18的结构图，大家可以参考一下"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8475d166-1de9-4fa7-b711-aa89fe5cc252",
   "metadata": {},
   "source": [
    "![resnet18](pic/6.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d2b27fb1-0ce6-4447-8a7e-c7858c368bfe",
   "metadata": {},
   "source": [
    "# 任务三 训练补全的VGG11和Resnet18"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "520bebc9-4ea5-47f4-ac44-54234bce2e38",
   "metadata": {},
   "source": [
    "<font size=4> 最后，对两个模型各训练3个epoch，看到训练结果和训练图线。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c1fc5c48-5a6f-4f61-8eaf-6bf4f083fe2b",
   "metadata": {},
   "source": [
    "# 任务四 补全LSTM模型"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0c3d5ffc-0f22-4728-a9bb-61d3558663c6",
   "metadata": {},
   "source": [
    "<font size=4> 补全LSTM的训练代码并进行训练"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7157b1cf-d30b-451e-bbc6-e18f3676411a",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "MindSpore",
   "language": "python",
   "name": "mindspore"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
